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Environment/Infrastructure 


►  The  environment  is  a  typical  high-visibility,  rapid 
fielding  government  ac(74//5/f/o/7  organization;  our 
process,  while  repeatable,  only  deals  with  “custom” 
articles  whose  lead  time  is  1  8+  months  per  item. 


Project  Manager 

(Military) 


DPM 

(Civilian) 


Improvement 

Area 


Sustainment 

Fleet  Mgmt 
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What  Needed  to  be  Done 


DoD  501 0.1 2-M  Procedures  for 
the  Acquisition  &  Management 
ofTechnicaiData,  May  1993: 

“An  organizational  element  shall 
be  assigned  responsibility  for 


monitoring  the  distribution  of 
contractor-prepared  data. 
Specifically,  procedures  shall 
be  established  to 

ascertain  actuai  receipt  of  the 
data  and  to 


ensure  that  aii  contractuai 
requirements  reiative  to 
technicai  data  have  been 
met . .  ” 


Lack  of  a  management  tool  &  a 
documented  process  in  CH-47F 
Product  Office  meant: 

•  No  predictive  analysis 

•  Lack  of  document  version  control 

•  Data  not  readily  available  when 
needed 

•  Unclear  document  status 

•  Redundancies 

•  Lack  of  cross-checks 

•  No  review  time  gates 

•  Lack  of  CDRL  “ownership” 

•  No  automatic  routing 

•  No  way  for  management  to  view 
status 


Time-intensive  reporting 


Why  ■  Issues 
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What  We  Did  -  2  -Pronged  Approach 


►  Tool:  CH-47F  DPM 

provided  direction  to 
CM/DM  Lead  (Mike)  to 
investigate,  evaluate,  and 
purchase  a  CDRL 
management  software  tool 


►  Process:  ch-47f  dpm 

directed  Cargo  Process 
Improvement  Lead  (Mamie) 
to  charter  a  process  action 
team  addressing  CDRL 
review 


Lean  Six  Sigma  Project 


Define 

1 

Measure  M  Analyze 

j^l  Improve  Control  j 
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Timelines 


2007 


2008 


2009 


o 

.o 


IMO  Tool 
Analysis 


DPM 

Direction 


(fi 

(fi 

<D 

O 

O 


★ 


Oct  Nov  Dec 

Jan  Feb  Mar  Apr  May  Jun  Jul  Aug  Sep  Oct  Nov  Dec 

Jan  Feb  Mar 

Initial  Data  Analysis  & 
Migration 

Remaining 
Data  Migration 

Kickoff  Meeting 

2  pilot  contracts 

CDRLvue 
Selection 
&  Purchase 


complete 


CDRLvue 
goes 


-vue 

“live” 


MY  Contract 
Award 


User  Training 


Expansion 
of  Scope 


Training  Package 
Development 


Working  Meetings  /  Process  Development 


Kickoff 

Meeting 


Process 

Document 

Complete 


i-IT 

PMO  Training 


Process  Staffing 


Process 

Approved 
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LSS  Project 


►  Project  Title:  Improving  the  Product  Line’s  Data  Item 
Deliverable  Response  Rate 

►  Definitions 

°  Contract  Data  Requirements  List  (CDRL)  -  DP  Form  1423: 

The  standard  format  for  identifying  potential  data 
requirements  in  a  solicitation  and  deliverable  data 
requirements  in  a  contract. 

°  Data  Item  Deliverable  (DID)  -  DP  Form  1 664:  A  form  that 
defines  the  intended  use,  preparation  instructions,  and 
content  and  format  requirements  for  a  specific  data 
product. 


The  Charter 


r^Defin^j 


►  Project  Description 

°  Improve  response  rate  to  the  customer  (vendor)  after 
internal  product  line  review. 

►  Business  Case 

°  Internal  Government  review  of  Vendor  (Contractor)  CDRLs 
were  exceeding  that  period  of  time  that  was  designated  in 
the  contract. 

•  The  lateness  of  responses  causes  delays  in  testing  efforts. 
suspension  of  sub-contractor  production,  and  impedes 
acquisition  of  long-lead  items. 

*  This  has  a  domino  effect,  slipping  an  already  tight  production 
and  fielding  schedule  of  a  critical  wartime  asset. 


Process  -  Receipt  &  Review  of  Data  Items 


Suppliers  =  Vendors 
(Contractors /OEMs) 


Vendor 

Submits 

CDRL 


Government 
receives,  logs 
&  distributes 
CDRL 


V 


Customers  =  Vendors 
(Contractors /OEMs) 


y 


f, 


Govt  reviews, 
integrates  & 
prepares 
letter  then 
forwards 


% 


Vendor 

Receives 

CDRL 

Disposition 


Inputs  =  CDRLs 

Analysis  Documents 
Contract  Documents 
Cost  Reports 
ECPs 
Plans 

Prod  Specs  (PD) 
Technical  Reports 
Test  Plans 


Outputs  = 

Disposition 
Recommendations 
Official  Letter 


The  product  line’s  response 
dates  are  specified  in  the 
contract  with  each  requirement 
having  a  unique  due  date. 
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Starting  Metrics 


Measure  |] 


Response  rates  to  the  customer  (vendor) 
after  internal  Government  review. 


Goal: 

Increase  percentage  rates  of: 
1 . 61-100  days  overdue, 

2.  31  -60  days  overdue; 

3.  0-30  days  overdue;  & 

4.  Pending 


05-Jan-09 


4, 7% 


Project  Sponsor/Owner  imposed  acceptable  limit: 
Upper  Requirement  Limit  <  1  20  days 


Distribution  Statement  A:  Approved  for' 
public  release;  distribution  is  unlimited. 


Note:  All  data  used  in  calculations  (not  sample  data) 
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Measure  || 

Starting  Metrics  (cont) 


►  Descriptive  Statistics 

“  Distribution  =  Non-Normal 
°  Mean  =  259 
°  Median  =  1 05 
°  Standard  Deviation  =  397.7 

►  Confidence  Interval  for  the 
Mean:  One-Sample  T,  Box 
Plot  &  Histogram 

°  Conclusion:  It  is  taking  too 
long  to  respond  to  the  data 
item  deliverable. 

►  Process  Capability  -  process 
is  NOT  capable. 

►  Process  Probability  -  low 
probability  that  we  can  make 
predictions. 
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Summary  for  1/  5/  09  Days  Overdue 


1 — 1 - 1 

1 _ 1 _ 1 

95%  Confidence  I  nterva Is 


Anderson-Darling  Normality  Test 
A-Squared  6.79 

P-Value<  0.005 


Mean 

StDev 

Variance 

Skewness 

Kurtosis 

N 


259.28 

397.74 

158198.96 

1.63472 

1.48026 

57 


Minimum 
1st  0  uartile 
Median 
3rd  0  uartile 
Maximum 


-85.00 

8.00 

105.00 

286.00 

1493.00 


95%  Confidence  I  nterval  for  Mean 
153.75  364.82 

95%  Confidence  I  nterval  for  Median 
32.00  143.02 

95%  Confidence  I  nterval  for  StDev 
335.80  487.93 


Boxplot  of  1/  5/  09  Days  Overdue 

(with  95%  t-confidence  interval  for  the  mean) 


400  600  800  1000 

y  51 09  Days  Overdue 


1 1 


Cause  &  Effect  Diagram 


Analyze  l) 


(or  Fishbone) 
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Color 

Root  Cause 

Old  data  in  database  & 

XYZ  Form 

Little  Management  Support 

Lack  of  Standard  Operating 
Procedure  (SOP)/Training 
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Action  Plan 


lmprove.il 


►  Root  Cause  -XYZ  Form 

°  Research  the  form  and  interview  individuals  to 
identify  the  “problem  areas”. 

°  “App  Code”  field 
°  “Authority”  field 
°  “Requiring  Office”  field 
°  “Remarks”  field 
°  “Distribution”  field 


Out-of-date,  but 
DoD  Mandated  Form 

Addressed  all  findings 
in  Form  Training/ 
Cheat  Sheet 


I 


►  Root  Cause  -  Old  data  in  database 

°  Perform  housecleaning  - 
reviewed  &  dispositioned  o 

every  single  CDRL 
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Improve  ij 


Action  Plan  (cont) 

►  Root  Cause  -  Little  Management  Support 

°  Develop  a  Dashboard 


Comparison  of  Responses  Overdue 

.1/5/2009  .2/7/2009  .3/6/2009  .4/10/2009 

□  Pending  00-30  031-60  061-100  0101-200  0201-400  *401  + 

D% 

47%,  19 

15%,6 

0^ 

C50 

15%,  6  15%,  6 

45%,  20 

11%,  5 

9%,  4 

18%,  8  13%,  6 

16%  10  25%,  15 

0^ 

to 

13%,  8 

1  1 1  6%, 

21%,  12 

16%,9  p 

U) 

7%,  4  21%,  12 

11%,  6 

O' 

%  20 

%  40 

%  60 

%  80 

1%  10( 
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Action  Plan  (cont) 

►  Root  Cause  -  Lack  of  Standard  Operating 
Procedure  (SOP)  /  Training 

°  Develop  a  Value  Stream  Map 
°  Voice  of  the  Customer  (VOC)  Questionnaires 
°  Draft  SOP  &  Training  Materials 
°  SOP  Training 


Value  Stream  Map 


CortiBctIn  PlooeHllti 

V9fKlcr{CUG!ain8i) 


Vendor  A 


Vendor  B 

Vendor  C 


VroduuL  LUI^ 
Tracking 


Tndependenr^ 

Organizations 
Analysis  Database 


Eeceive 
otification 
Letter  & 
Data  item 


hput  data 
Infto 

Database 


^  A 


Email  data 
Item  to 
Reviewers 


Forward 
Commenfts 
To  Lead 


Incorporate 

comments 


'orward 
b  PA 


PA 

Wi-ites 

letter 


'  !■* 


I  ggi 


>  !■> 


1 


'  !■ 


Vendor  A 


t 

1 

I 


Letter  to  Letter  to 

Bus  Chief  customer 

■ . . . . . .  ■  ■  ■  ■  ^ 


120^  nfUjIn  ^  ragpi^mln  <pi|fn  9Bfl^  ^  ^nln  2^pHiiiln 


611*1 


10  nil 


611*1  2«Dii*i  611*1  90ii*i  611*1  4611*1  611*1  611*1 


611*1 


80inln 


[  Moderate 
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Indication  of  time  being 
spent  on  each  task 


I  Process  Cycle  Time 

12,980  minutes 

216  hour3| 

“|5. 4  work  weeks  | 

Process  Time 

_ 

2,R4n  miniJtRfi 

44  hours \ 

1.1  work  weeks 

Whit  ihould  bt  tiking  1 .1  work  intki  (or  6  j5  diyi) 
li  tiking  64  work  wtoki  (27  diyi). 
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Ending  Metrics 

►  Descriptive  Statistics 

°  Distribution  =  Non-Normal 
»  Mean  =  57 
°  Median  =  1  0 

°  Standard  Deviation  =  107.3 

►  Confidence  interval  for  the 
Mean:  One-Sample  T,  Box  Plot 
&  Histogram 

°  Conclusion:  It  is  taking  too 
long  to  respond  to  the  data 
item  deliverable. 

►  Process  Capability  -  process  Is 
NOT  capable. 

►  Process  Probability  -  low 
probability  that  we  can  make 


Improve  l| 


Summary  for  4/ 10/  09  Days  Overdue 


-CZL 


95%  Confidence  I  ntervals 

Mean-  | - • - 1 

Median-  | - • - 1 

- 1 - 1 - 1 - 1 - r-* 

0  25  50  75  100 


Anderson- Darling  Normality  Test 

A -Squared 

3.08 

P-Value  < 

0.005 

Mean 

56.900 

StDev 

107.310 

Variance 

11515.528 

Skewness 

0.939333 

Kurtosis 

-0.661762 

N 

40 

Minimum 

-59.000 

1st  Q  uartile 

-20.750 

Median 

10.000 

3rd  0  uartile 

151.000 

Maximum 

269.000 

95%  Confidence  1  nterval  for  Mean 

22.580 

91.220 

95%  Confidence  1  nterval  for  Median 

-15.768 

44.481 

95%  Confidence  1  nterval  for  StDev 

87.905 

137.790 

Boxplot  of  4/  10/  09  Days  Overdue 

(with  95%  t-confidence  interval  for  the  mean) 
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[l  Improve  l| 


Descriptive  Statistics  Comparison 


►  Conclusion:  The  Mean,  Standard  Deviation  and  Variance  have 
all  been  decreasing  throughout  the  efforts  per  the  data 
collected  and  compared  below,  but  the  process  remains 
incapable,  unstable,  and  unpredictable.  There  is  still  a  high 
level  of  variability  in  the  process. 


Variable 

1/5/09  Days  Overdue 
2/7/09  Days  Overdue 
3/6/09  Days  Overdue 
4/10/09  Days  Overdue 
5/13/09  Days  Overdue 


Total 

Count 

57 

72 

45 

40 

47 


SE  Mean 


Variable 

Q1 

Median 

Maximum  ^ 

1/5/09  Days  Overdue 

8.0 

105.0 

286.0 

1493.0 

1.63 

1.48 

2/7/09  Days  Overdue 

-3.0 

71.5 

154.3 

1460.0 

2.03 

3.07 

3/6/09  Days  Overdue 

O 

CM 

CM 

1 

8.0 

166.5 

289.0 

0.76 

1 

O 

00 

4/10/09  Days  Overdue 

-20.8 

10.0 

151.0 

269.0 

0 . 94 

-0.66 

5/13/09  Days  Overdue 

-13.0 

12.0 

51 . 0 

302 . 0 

1 . 73 

2 . 00 
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Decreasin 


Sustaining 

►  Utilize  Dashboard 

►  SOP  &  Form  Training  completed 

►  Cheat  Sheet  distributed 

►  Control  Plan  put  in  place 

►  Data  collection  continues 


The  Results 


►  Reduced  the  average  late 
response  rate  from  ~260 
days  to  ~  42  days  in  6  mo. 


►  A  large  reduction  in  the 
backlog  of  data  item 
deliverable  responses 


►  Captured  a  one-time  cost 
avoidance  of  -$623,000. 


►  Reduced  waste  and 
increased  throughput 


Average  Late  Response  Rate 


Objective 


►  Improved  organizational 
capability  to  operate  more 
efficiently  and  effectively. 


Warm  &  Fuzzy 
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The  Results  -  Calculations 


Every  how 

Time  it  takes 

total  per 

many  days 

(manhours) 

$$ 

1 0  days 

Updating  Schedule 

10 

N 

$N 

$ 

DM  sends  Reminders 

10 

N 

$N 

$ 

Reviewers  (3)  Re-Education 

20 

N 

$N 

$ 

total  $NNN 


1/5/09  Days  Overdue  / 10  days  +  ~  $647,318 
4/10/09  Days  Overdue  / 10  days  =  ~  $23,722 


Challenges 

►  Installation  of  a  new  integrated  software  system 
to  manage  data  deliverables; 

►  Regulatory  requirements  to  operate  within  strict 
DoD  Acquisition  processes  and  guidelines; 

►  An  out-of-date,  but  DoD  mandated  form; 

►  Need  of  an  organizationai  Standard  Operating 
Procedure  (SOP);  and 

►  Lack  of  an  effective  means  of  reporting  review 
and  response  status  to  the  management. 

►  Will  not  realize  any  benefits  from  SOP  &  Form 
Training  for  1 2+  months 


What  Did  We  Learn 


►  Need  to  make  improvements  personal  -  how 
does  it  affect  “me” 

►  Improvement  has  a  domino  effect 

►  Improvements  build  on  each  other 

►  Need  for  closely  managing  the  pace  of 
change 

►  Commonality  &  consistency 

►  Importance  of  training  (tool  &  pr( 
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Contact  Information 


►  Mamie  Danley  Morgan 

CH-47  Process  Improvement  Lead,  Black  Belt 
QinetiQ  North  America 
(256)  31  3-0716  DSN:  897-0716 

mamie.morqan@us.armv.mil 

►  Mike  Cowperthwait 

CM /DM  Lead 

Cargo  Helicopters  Project  Mgmt  Office 
(256) 876-4583  DSN  746-4583 

BB:  (256)  509-6380  Fax:  (256)  31  3-1  91  5 
michael.cowperthwait@us.armv.mil 
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